Repository navigation
fix(transport): close upstream work once on client disconnect - #31
Conversation
Give Flask streaming iterators and the Worker roleplay transport a single cancellation owner, so a disconnect, read error or completion releases the upstream exactly once and is recorded as an ambiguous outcome with unknown usage. A failed SSE socket read now ends the stream instead of re-reading the partly consumed response.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
multillm-proxy | 6d34713 | Oct 08 2026, 09:59 PM |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d34713a77
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| response.close = context.close | ||
| if isinstance(response, Response): | ||
| source = response.response | ||
| response.response = CancellationIterator( | ||
| lambda: iter(source), context, close_source=getattr(source, "close", None), |
There was a problem hiding this comment.
Mark fully consumed raw responses as complete
For unified requests returning a requests.Response, this replaces close() but only wraps Flask Response bodies. The raw passthrough path in stream_upstream_response() therefore consumes the complete body and invokes this overridden close() from its finally, which records cancelled with ambiguous=true instead of completion. Normal raw chat, image, and protocol responses will consequently expose an uncertain cancellation outcome even when every byte was delivered; the raw-body iterator needs to signal complete() on EOF before cleanup.
Useful? React with 👍 / 👎.
| def complete(self) -> None: | ||
| self._finish("complete", classify_upstream_outcome(200)) |
There was a problem hiding this comment.
Classify completion using the actual response status
Every exhausted response is classified as HTTP 200 here, regardless of its real status. For example, fully consuming an owned 401, 429, or 503 response produces an UpstreamOutcome reporting provider success and accepted credentials, contradicting the repository's status classification and potentially corrupting health or credential accounting once this callback is consumed. Capture the response status when binding and classify completion with that value.
Useful? React with 👍 / 👎.
| close(reason = "cancelled") { | ||
| if (this.settled) return Promise.resolve(); | ||
| this.finish(reason === "upstream_header_timeout" ? reason : "cancelled"); |
There was a problem hiding this comment.
Preserve interrupted outcomes when closing Worker streams
When an upstream body read throws, wrapResponse() explicitly calls close("interrupted"), but this expression rewrites every reason except the header timeout to cancelled. The same misclassification occurs for transport failures cleaned up after fetch rejection, so observers cannot distinguish provider/socket interruption from a client cancellation even though reason is part of the new outcome contract. Preserve interrupted rather than collapsing it to cancelled.
Useful? React with 👍 / 👎.
| return Promise.resolve(); | ||
| } | ||
|
|
||
| cleanup() { void this.close(); } |
There was a problem hiding this comment.
Keep cleanup compatible with subsequent body cancellation
The new cleanup() now errors the wrapped downstream stream via close(), so callers that intentionally discard a response by calling cleanup() and then awaiting response.body.cancel() receive a rejected AbortError. This affects the invalid-content-type branches in worker/roleplay/continuation.mjs and worker/roleplay/nonstream-repair.mjs: instead of returning null and handling an unusable continuation, they propagate an exception and can fail the entire roleplay response. Make cleanup leave an explicitly discarded body cancellable, or update those cleanup sequences so intentional disposal cannot reject.
Useful? React with 👍 / 👎.
| context = getattr(response, "multillm_cancellation", None) | ||
| if isinstance(context, CancellationContext): | ||
| return context |
There was a problem hiding this comment.
Attach observers when rebinding an owned response
If a response already has a cancellation context, this returns it without applying the newly supplied on_outcome. Unified dispatch now binds every response without an observer, so a later accounting or settlement layer calling the documented bind_cancellation(response, on_outcome=callback) API will silently receive the existing context and its callback will never run. Preserve or register the new observer when reusing a context, or arrange for the initial unified binding to receive it.
Useful? React with 👍 / 👎.
9e456fc
into
integrate/gateway-waves-1-2
Summary
Closes upstream work exactly once when a client disconnects, and records the result as uncertain rather than free or successful. This is a cleanup fix with no flag. Completed responses keep their exact bytes, status codes and headers.
iter_stream_contentanditer_stream_linesnow return closable iterators that own the upstream response from the start. A client disconnect, a read error, normal exhaustion or a repeated close releases the upstream once.worker/upstream-cancellation.mjs.worker/roleplay/transport.mjsunder 1,000 lines.Details are in
docs/request-cancellation.md, which also lists the paths not yet covered.Integration
The Wave 3 integration PR connects these outcomes to request accounting. It also binds the provider-specific streaming generators in
services/proxy_service.py, and passes the request signal to other native Worker provider fetches.Shared registrations
tests/test_request_cancellation_worker.mjstotest:worker.worker/upstream-cancellation.mjsto the secret firewall inventory.tests/helpers/load_cloudflare_worker.mjsnow resolves the new transport import.Tests
tests/test_request_cancellation.pyandtests/test_request_cancellation_worker.mjs: new.npm run test:worker: 1,239 passed, including the existing roleplay transport suites.cf:migrations:check,test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.tests/test_frontend_build.mjsfails locally because@tailwindcss/cliis not installed. It fails the same way on main.This is part of Wave 3. It is based on the Wave 2 integration branch (#25), and the Wave 3 integration PR adds the README line.